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^ , Abstract. This paper is devoted to present a method of proving verification theorems 

for stochastic optimal control of finite dimensional diffusion processes without control in the 
diffusion term. The value function is assumed to be continuous in time and once differentiable 
in the space variable (C ' 1 ) instead of once differentiable in time and twice in space (C 1 ' 2 ), 
like in the classical results. The results are obtained using a time dependent Fukushima - 

rS^ Dirichlet decomposition proved in a companion paper by the same authors using stochastic 

calculus via regularization. Applications, examples and comparison with other similar results 
are also given. 
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1 Introduction 



5_^ 

In this paper we want to present a method to get verification theorems for stochastic optimal 

control problems of finite dimensional diffusion processes without control in the diffusion 

term. The method is based on a generalized Fukushima - Dirichlet decomposition proved 

in the companion paper |22j . Since this Fukushima - Dirichlet decomposition holds for 

functions u : [0, T] x W 1 — ► M. that are C° in time and C 1 in space (C 0,1 in symbols), our 

verification theorem has the advantage of requiring less regularity of the value function V 

than the classical ones which need C 1 regularity in time and C 2 in space of V (C 1 ' 2 in 

symbols), see e.g. Q31 pp. 140, 163, 172]. 

There are also other verification theorems that works in cases when the value function 
is nonsmooth: e.g. it is possible to prove a verification theorem in the case when V is only 
continuous (see [221, [27j, |23 Section 5.2]) in the framework of viscosity solutions. However 
all these results applied to our cases are weaker than ours, for reasons that are clarified in 
Section El 

Since the method is a bit complex and articulated we present first in next Section 
the statement of our verification Theorems 12.71 and 12.81 in a model case with simplified 



assumptions (which substantially yield nondegeneracy of the diffusion coefficient). In the 
same section we also put Subsection 12 . II where we give some comments on the theorem, its 
applicability and its relationship with other similar results. 

Below we pass to the body of the paper giving first some notations in Section [3] and 
then presenting the general statements (including also possible degeneracy of the diffusion 
coefficient) and their proof in Section 0J Section is devoted to necessary conditions and 
optimal feedbacks. 

SectionsElandHcontain applications of our technique to more specific classes of problems 
where other techniques are more difficult to use. The first is a case of exit time problem 
where the HJB equation is nondegenerate but C 1 ' 2 regularity is not known to hold due to 
the lack of regularity of the coefficients; the second is a case where the HJB equation is 
degenerate parabolic. Finally in Section [S] we compare our result with other verification 
techniques. 

2 The statement of the verification theorems in a model 
case 

To clarify our results we describe briefly and informally below the framework and the state- 
ment of the verification theorem that we are going to prove in a model case. The precise 
statements and proofs are given in Section QJ then in Sections El 13 applications to more 
specific (and somehow more difficult) classes of problems are given. We decided this struc- 
ture since it is difficult to provide a single general result: we can say that we introduce a 
technique, based on the Fukushima-Dirichlet decomposition and on its representation given 
in (2217 that can be adapted with some work to different settings each time with a different 
adaptation. 

First we take a given stochastic basis ($7, (J- S ) s>0 , F) that satisfies the so-called usual 

conditions, a finite dimensional Hilbert space A = R™ (the state space), a finite dimensional 
Hilbert space E = R m (the noise space), a set U C R fe (the control space). We fix then a 
terminal time T £ [0, +00] (the horizon of the problem, that can be finite or infinite but is 
fixed), an initial time and state (t, x) £ [0, T] x A (which will vary as usual in the dynamic 
programming approach). The state equation is (recall that % = [t,T] n K) 

dy(s) = [Fo(s,y(s)) + F 1 (s,y(s),z(s))]ds + B{s,y(s))dW{s), s&%, 

y(t) = x, (1) 

where the following holds (for a matrix B by ||S|| we mean ^,- \b{j\ and, given E, F finite 
dimensional spaces, by C (E, F) we mean the set of linear operators from E to F). 

Hypothesis 2.1 1. z : % x Q — > U (the control process) is measurable, locally integrable 
in t for a.e. to £ fi, adapted to the filtration (-? ? s) s >o;' 

2. F Q : T x R" -» W\ F 1 : T x R" x U -> R™ and B : % x W l -> C(E,R n ) are 
continuous. 

3. There exists C > such that Vt £ To, Viei, xi £ R™, 

KF (t,afi)-Fo(t,s 2 ),a;i-a;2>| + ||B(t,a:i)-B(t,S2)|| 2 < C\x 1 ~x 2 \ 2 1 

|(Fo(t,s),*)| + ||B(t,a;)|| 2 < C 

and there exists a constant K such that, Vi £ T), Vx £ R™, Vz £ U , 

\F\ (t, xi,z) — Fi (t,x 2 ,z)\ < K\xi-x 2 \ 

\{F 1 {t,x,z),x)\ < K (1 + \x\ + \z\) . 



1 + kl 2 



4- For every x E 



\F*(t,x)\ + \\B{t,x) 
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dt< 



We call Z a< i (t) the set of admissible control strategies when the initial time and state is 
(t, x) defined as 

, . _ \ z : % X O — ► U, z measurable, adapted to (•? r s) s >o > 
1 locally integrable in s for a.e. cj E £1 

and call y (s; £, x, z) the state process associated with a given z E iJ a£ 2 (£); this is the strong 
solution of the equation Q) which exists and is unique for any given z thanks to Theorem 
1.2 in 25, p.2]. 

Consider now the case T < +oo. We try to minimize the functional 

J(t,x;z) =E< / l(s,y(s;t,x,z),z(s))ds + (j)(y(T-,t,x,z))} (2) 



over all control processes z E Z a d it) . On the coefficients we assume the following. 

Hypothesis 2.2 / : [0, T] x R™ x U — ► R and : R™ — > R are continuous and such that 
for each z £ Z ac i(t) the function (s,co) — * l(s,y (s,uj) ,z(s,u>)) is lower semiintegrable in 
[t, T] X Q (recall that a real valued function f is lower semiintegrable in a measure space 
(M, (j,) if J M f~d/J, < +oo, where f~ stands for the negative part of f i.e f~ = (— /) V 0). 

Remark 2.3 The above requirement implies that J(t,x;z) is well defined and > — oo for 
each z 6 Z ac i(t). It is assumed explicitely to cover a more general class of problems. This 
is obvious under additional assumptions, e.g. if I and 4> are bounded below. However some 
interesting problems arising in economics contain functions I like —In z and in this case 
the lower semiintegrability of (s,uj) — ► — lnz(s,w) for each z E Z a d{t) follows by ad hoc 
arguments. These are problems with state constraints that are not explicitely treated in this 
paper but we want to set our framework so to be able to treat such class of problems. fl 



The value function is defined as 

V (t, x) = inf J (t, x; z) (3) 

zeZad(t) 

and a control z* E Z a d (t) is optimal at (t,x) if V(t,x) = J (t,x; z*). The current value 
Hamiltonian is defined, for (t,x,p,z) E [0,T] x R™ x R n x U as 

H C v {t, x,p;z)= (F (t, x) ,p) + (Fi (t,x, z) ,p) + I (t,x, z) 

and the (minimum value) Hamiltonian as 

H(t,x,p)= inf H C v (t,x,p;z) . 

z£U 

Since the first term of Hcv (t, x,p; z) does not depend on the control z we will usually define 
Hcv (*) x , P; z ) = (Fi (*> x,z),p) +1 (t, x, z) 

and 

H°(t,x,p)=m£H° v (t,x,p;z), (4) 

z£U 



so we can write 

H C v(t,x,p;z) =: (F (t,x),p) + H^ v (t,x,p;z) (5) 

and 

H(t,x,p)=: (Fo (t, x) , p) + H° (t, x,p). (6) 

When T < +00 the Hamilton-Jacobi-Bellman (HJB) equation for the value function is a 
semilinear parabolic PDE ((t,x) e [0,T] x R") 

-d t v (t, x) = -Tr [B* (t, x) d xx v(t, x)B (t, x)} + (F (t, x) , d x v (t, x)) + H° (t, x, d x v (t, a;)) , 

(7) 
with the final condition 

v{T,x)=(j>{x), ier. (8) 

To ensure finiteness and continuity of the Hamiltonian we need also to add the following 
assumption. 

Hypothesis 2.4 The functions I, F± and the set U are such that the value function is 
always finite and the Hamiltonian H°(t,x,p) is well defined, finite and continuous for every 
(t,x,p) S [0,T] xl" x R™. 



Remark 2.5 The above Hypothesis \2.J\ is satisfied e.g. when l,(/> are continuous and U is 
compact. Another possibility is to take U unbounded, F\ sublinear, I (t, x,z) — g (x) + h (z) 
with g continuous and bounded, h continuous and such that 

\h{z)\/\z\ — > +cxd as \z\ — > +00; 

this case was studied e.g in pllifll 1^7) ^ even in the infinite dimensional case (see on this the 
Sections®^. I 

The statement of the classical verification theorem for this model problem when T < +00 
is the following, see Definition 14. 51 for the definition of strict solution of equation 0-ljSJl. 

Theorem 2.6 Assume that Hypotheses HH1 HTl Wl\ hold true. Let v e C 1 ' 2 ([0,T] x R rl ) 
be a polynomially growing strict solution of the HJB equation fTP-ifSp on [0, T] x R™. Then 
the two following properties hold true. 

(i) v< V on [0,T] xl"; 

(ii) fix (t, x) £ [0, T] x R"; if z S Z ac i (t) is such that, calling y (5) = y (s; t , X, z) , 

H° (s, y (s) , d x v (s, y (s))) = H^y (s, y (s) , d x v (s, y (s)) , z (s)) , F-a.s., 

for a.e. s € [t, T], then z is optimal at (t, x) and v (t, x) — V (t, x). 

This theorem states a sufficient optimality condition and its proof is based on Ito's 
formula, see e.g. |H9I p. 268]. In the classical context also necessary conditions and existence 
of optimal feedback can be proved, see again (J2J1 p. 268]. 

The statement of our Verification Theorem in the model problem described above is very 
similar. We give it in the case when the following nondegeneracy hypothesis hold 

B^ 1 (t, x) Fi (t, x, z) is bounded on [0, T] x R" x U, (9) 

leaving the discussion for the general case to Section^] (strong solutions of the HJB equation 
0-© are defined in Definition Q|) . 



Theorem 2.7 Assume that Hvvotheses Wl\ HT1 \H\ and 0) hold true. Letv £ C ' 1 ([0,T] x I 
be a polynomially growing strong solution of the HJB equation Q)-@) on [0,T] x R™. Then 
(i) and (ii) of Theorem \2.b\ above hold. 

In fact we will go further: 

• proving a result in the case when © does not hold; 

• showing that in our setting we can obtain a necessary condition and the existence of 
optimal feedback controls on the same line of the classical results, see Section 03 

Moreover in a paper in preparation we will show that also some cases where the drift 
is a distribution can be treated and also cases where the solution of HJB enjoys weaker 
regularity. 

When T = +oo, Fq, Fi and B do not depend on time, we consider the problem of 
minimizing 

J (£, x; z) = E < / e~ l l\ (y (s;t,x, z) ,z (s)) ds 

where I (t, x, z) = e~ xt l\ (x, z), with A > 0, l\ continuous and bounded. In this case the value 
function V is defined as in J2J but its dependence on t becomes trivial as V (t, x) — e xt V (0, x) 
for every t and x. Then we set t — and call Vq (x) — V (0, x). The HJB equation for Vq (x) 
becomes an elliptic PDE (x E R") 

Xv (x) = ^Tr [B* (x) d xx v(x)B (x)] + (F (t, x) , d x v (t, x)) + H 1 (x, 3 x v (t, x)) , (10) 



where 



H 1 (x,p) = inf {(Fi (x,z) ,p) + h (x,z)} =: inf H^ v (x,p;z). (11) 

z£(7 z£{7 



In this elliptic case the statement of our verification theorem is the following (for the 
definition of strong solution of the HJB equation (|10|l see Definition 14.14(1 . 

Theorem 2.8 Assume that Hvvotheses \2.1\ and 0) hold. Assume also that I \ is continuous 
and bounded and that Hypothesis \2-4\ hold true with H 1 in place of H°. If v is a bounded 
strong solution of the HJB equation (JOjj and v E C 1 (R ra ) then v < Vq. Moreover fix x E M. n ; 
if z E Z a d (0) is such that, calling y (s) — y (s; 0, x, z), 

H 1 (V i s ) , 9 x v (s, y (s))) = H x cv (y (s) , d x v (s, y (s)) , z (s)) , for a.e. s> 0, P-a.s., 

then z is optimal at (0,x) and v (x) — Vq(x). 

2.1 Other Verification Theorems for stochastic optimal control prob- 
lems 

We recall that in the literature on stochastic optimal control other verification theorems for 
not C 1,2 functions were proved. Roughly speaking we can say that each of them (and the 
technique of proof, too) is strictly connected with the concept of weak solution of the HJB 
equation that is considered. We have in fact the following. 

• (Strong solutions). In |18U21j (see also j3^JEj) m a setting similar to ours but in 
infinite dimension, a verification theorem is proved assuming that there exists a strong 
solution v of HJB which is C 1 in space. In fact the method outlined here goes in the 



same direction, but improves and generalizes in the finite dimensional case the ideas 
contained there. Such improvement is not a straightforward one, as we need to use 
completely different tools to get our results and we get more powerful theorems. We 
refer to Sections and [7] for examples and to Section [S] for explanations. 

• (Viscosity solutions). In |271 1391 l2*3"| a verification technique for viscosity solutions is 
introduced and studied. Such technique adapts to the case when v is only continuous 
and so is very general and applicable to cases when the control enters in the diffusion 
coefficients B. However in the case of our interest, i.e. when v is C 1 in space, such 
technique gives weaker results as it requires more assumptions on the coefficients Fq, 
F\, B and on the candidate optimal strategy; see Section [S] for explanations. 

• (Mild solutions). In [TH Theorem 7.2] a verification theorem is given when the HJB 
equation admits mild solutions which are C 1 in space (see Definition 16 . 131 or Remark 
17.21) and when the solution can be represented using the solution of a suitable backward 
SDE. The results available with this technique are limited to infinite dimensional cases 
where mild solutions exist and Girsanov theorem can be applied in a suitable way. Such 
restrictions prevent the use of this technique (at least to the present state of the art) 
e.g. in the cases described in Sections HO and [7| see Section [S] for explanations. 

Moreover we want to stress the fact that the Fukushima-Dirichlet decomposition we got 
in |22| is in fact stronger than what we need to prove the verification theorems above. In fact 
for such purpose it would be enough to prove a Dynkin-type formula (roughly speaking an 
Ito formula after expectation) which is in general easier. We decided to state and prove such 
Fukushima-Dirichlet decomposition since it can help to deal with stochastic control problems 
where the criterion to minimize does not contain expectation (e.g. pathwise optimality and 
optimality in probability) . In such context the HJB equation becomes a stochastic PDE and 
to get a verification theorem, Dynkin-type formulae cannot be used. See e.g. |l(JII28ll2^1l33 | 
on this subject. 

To sum up we think that the interest of our verification results is the fact that 

• we obtain them using only the fact that the solution v of HJB belongs to C 0,1 and not 
necessarily to C ' obtaining better results than other techniques. 

• they can be applied also to problems with pathwise optimality and optimality in prob- 
ability. 

Remark 2.9 As a first step of our work, we consider here the case of a stochastic optimal 
control problem in finite dimension with no state constraints. We are aware of the fact that 
in many cases, the HJB equation associated with the control problem admits a C > solution, 
so that our method does not give a real advantage in this case. However 

• in some cases, mainly degenerate ones (see Sections and HI) it is known that the 
solution of HJB is C° (or C a with Holder exponent a > 0) in time and C 1 in space 
but C ' regularity is not known at this stage. In particular we show an explicit example 
where the value function is C 1 but cannot be C 2 in space; 

• we think that such technique could be extended also in cases where HJB equation is 
intended in a generalized sense; in a paper in preparation, we investigate a verifica- 
tion theorem related to an SDE whose drift is the derivative of a continuous function 
therefore a Schwartz distribution, see for instance ]12$ - That state equation models a 
stochastic particle moving in a random (irregular) medium; 



• we consider problems without state constraints; again we think that some state con- 
straints problems can be treated with our approach but we do not handle them here: we 
provide an example with exit time which usually presents similar difficulties; 

• we think that our method can be extended to the infinite dimensional case, where C 1,2 
regularity results for the solution of HJB equation are much less known while C 1 reg- 
ularity can be found in a variety of cases (see e.g. Jb\\18l Wf\ Yfl). We do not perform 
this extension here leaving it for further work. I 



Remark 2.10 Similar ideas as in this work have been expressed with a different formalism 
in J$j. There the authors have implemented a generalized ltd formula in the Krylov spirit for 
proving an existence and uniqueness result for a generalized solution of Bellman equation 
for controlled processes, in a non-degeneracy situation. I 



3 Notations 

Throughout this paper we will denote by (f2, T , P) a given stochastic basis, where T stands 
for a given filtration (J- S ) s>0 satisfying the usual conditions. Given a finite dimensional real 
Hilbert space E, W will denote a cylindrical Brownian motion with values in E and adapted 
to {Ts) s>0 . Given < t < T < +oo and setting % = [t,T] n K the symbol C? {% x Q; E), 
will denote the space of all continuous processes adapted to the filtration T with values 
in E. This is a Frechet space if endowed with the topology of the uniform convergence in 
probability (u.c.p. from now on). To be more precise this means that, given a sequence 
(X n ) C Cjr {% x O; E) and X e C r (T t x SI; E) we have 



X n ->X 



if and only if for every e > 0, t\ € %. 



lim sup P (\X% - X S \ E > e) = 0. 

n-H-oo s£ [ tjtl ] 

Given a random time t > t and a process {X s ) s ^q- t , we denote by X T the stopped 
process defined by XJ = X sA%T . The space of all processes in [t, T], adapted to T and square 
integrable with values in E is denoted by Ljr(t,T;E). S n will denote the space of all 
symmetric matrices of dimension n. 

Let k G N. As usual C k (W 1 ) is the space of all functions : W l — ► M. that are continuous 

together with their derivatives up to the order k. This is a Frechet space equipped with the 

seminorms 

sup |u(a;)| R + sup \d x u(x)\ Rn + sup \d xx u (x)| M „ x „ + ... (12) 

xeK xeK xgk 

for every compact set K CC K™. This space will be denoted simply by C k when no confusion 
may arise. If K ia compact subset of M. n then C k (K) is a Banach space with the norm l|12() . 
The symbol C k (R™) will denote the Banach space of all functions from M. n to R that are 
continuous and bounded together with their derivatives up to the order k. This space is 
endowed with the usual sup norm. Passing to parabolic spaces we denote by C° {% x W 1 ) 
the space of all functions 



that are continuous. This space is a Frechet space equipped with the seminorms 

sup \u(s,x)\ M 
(s,ic)e[4,ti]x.ff 

foreveryti > and every compact set K CC R"). Moreover we will denote by C 1 ' 2 (T t xR") 
(respectively C 0,1 (T t x R n )), the space of all functions 

u:T t xR n ^R, (s,x)^u(s,x) 

that are continuous together with their derivatives dtu, d x u, d xx u (respectively d x u). This 
space is a Frechet space equipped with the seminorms 

sup \u(s,x)\ M + sup \d s u(s,x)\ Rn 

(s,x)e[t,t l \y.K (s,x)£[t,t 1 ]xK 

+ sup \d x u(s,x)\ mn + sup \d xx u(s,x)\ M „ x „ 

(s,i)e[(,(i]xif (s,x)e[t,t!]xK 

(respectively 

sup \u(s,x)\ R + sup \d x u(s,x)\ Rn ) 

(s,x)e[t,ti]xK (s,x)e[t,ti]xK 

for every t\ > and every compact set K CC R™. This space will be denoted simply by 
C 1,2 (respectively C ' 1 ) when no confusion may arise. 

Similarly for a,0 € [0, 1] one defines C Q ' 1+/3 {% x R") (or simply C Q - 1+/3 ) as the sub- 
space of C 0,1 (7t x R") of functions u : % X R™ ^^ R such that are u (•, x) is a— Holder con- 
tinuous and d x u (s, •) is /3— Holder continuous (with the agreement that 0-H61der continuity 
means just continuity). If such properties hold just locally then such space is denoted by 
C ioc +P ( T t x R ™)- Similarly given an open subset of O of R" one can define C Q < 1+/5 {% x O) 
and C / "' c 1+/3 {T t x O). If if is a compact subset of % x R" we define C k (K) or C Q ' 1+/3 (K) 
as above. Similarly to Cj; (R n ) we define the Banach spaces C£ {% x W 1 ) C b 1,2 (T t x R"), 
C"' 1+/3 (T t x R"), C 6 0,1 (T t x K"). 

4 Proof of the verification theorems in the model case 

In this section we give the precise statement and the proof of the verification Theorems 12. 71 
and 12.81 for the model problem described in Section [3 Then in Sections El and El we will 
consider two families of problems to which our technique apply. 

Consider first the parabolic case fixing the horizon T < +oo. Under Hypotheses 12. II l2~2l 
and 12.41 we seek to minimize the cost J (t, x; z) given in @ over all admissible z e Z a d (t) 
where the state equation is given by Q. 

We define the operator 

C : D (Co) C C° ([0, T] x R") — > C° ([0, T] xl"), D (£ ) = C 1 ' 2 ([0, T] x R") , 

C v (t, x) = d t v (t, x) + (F [t, x) , d x v (t, a?)> + -Tr [B* (t, x) xx v (t, x) B (t, x)} . 
The HJB equation associated with the problem Q - J2J can then be written as 

Cov{t 7 x)+H a (t,x,d x v{t,x)) = 0, v(T,x) = cj>(x), (13) 

where H° is given in Q. 



Now we want to apply the representation result proved in 22 Section 4, which we recall 
below for the reader's convenience. First we consider the following Cauchy problem for 
heC°([0,T\ xK"). 

Cqu (s, x) = h (s, x) , u (T, x) = 4> (x) , (14) 

with the following definitions of solution. 

Definition 4.1 We say that u G C° ([0, T] x R") is a strict solution to the backward Cauchy 
problem \H\j if u G D (Cq) and \14\j holds. 

Definition 4.2 We say that u G C°([0,T] xl") is a strong solution to the backward 
Cauchy problem \14\j if there exists a sequence (u n ) C D (Cq) and two sequences (</>„) C 
C° (W 1 ), (h n ) C C^i^T] x R n ), such that 

1. For every n £ N u n is a strict solution of the problem 

C Q u„ (t, x) = h n (t, x) , u n (T, x) = (f) n (x) . 

2. The following limits hold 

u n — >u inC°([0,T] xl"), 
h n — >h inC°([0,T] xl"), 
4> n — > (/) in C° (R n ) . 

The representation result is the following (Corollaries 4.6 and 4.8 of |22j\ 

Theorem 4.3 Let 

6i : [0,T] xl" xSJ^l", 

be a continuous progressively measurable field (continuous in (s, x)) and 

b : [0, r]xl"^ R", a : [0, T] x R™ -> £ (R m , R n ) , 

&e continuous functions. Let u £ C ' 1 ([0, T] xl") 6e a strong solution of the Cauchy 
problem \14\ l- 

Fix t G [0,T], x G R™ and let (S s ) be a solution to the SDE 

dS s =b 1 (s,S s )ds + a (s 1 S s )dW s , S t — x. 

Then 

u(s,S s ) — u(t,St)+ / h(r, S r )dr + / (d x u(r,S r ) ,b\ (r, S r ) — b(r, S r )) dr 



+ / d x u{r,S r )a(r,S r )dW r . 

provided that: either we can choose the approximating sequence (u n ) of Definition \J~2] so 
that for every < t < s < T 

lim / {d x u n (r,S r )-d x u(r,S r ),b 1 (r,S r )-b(r,S r ))dr = 0, u.c.p., (15) 



n — ^ + oc 



or the function 

(£, x, uo) —* a^ 1 (t, x) \b\ (r, x,ui) — b (r, x)] , 

(where a~~ l stands for the pseudo-inverse of a), is well defined and bounded on [0, T] x R" x Q. 



Remark 4.4 If 



lim d x u n = d x u, in C° ([0, T] x R") . 

n — > + oo 



then Assumption H5}) is verified. This means that the result of Theorem \4.3\ above applies 
if we know that u is a strong solution in a more restrictive sense, i.e. substituting the point 
2 of Definition ^. 2\ with 

u n — >u in C°([0,T] xl"), 
a A ^a i «mC»([0,T] xl»), 
h n — >h mC a ([0,T] xl"), 
<p n — > <j> in C° (R™) . 

This is a particular case of our setting and it is the one used e.g in 11 Ml \21f to get the 
verification result. We can say that in these papers a result like Theorem \4-3\ is proved 
under the assumption that u is a strong solution in this more restrictive sense. It is worth to 
note that in such simplified setting the proof of Theorem \4.S\ follows simply by using standard 
convergence arguments. In particular there one does not need to use the Fukushima-Dirichlet 
decomposition presented in Section 3 of ]2itf . So, from the methodological point of view there 
is a serious difference with the result of Theorem \4-3l see Section^ for comments. | 

To apply Theorem 14.31 to our case we need first to adapt the notion of strong solution 
and to rewrite the main assumptions. Let us give the following definitions. 

Definition 4.5 We say that v G C° ([0, T] x R n ) is a strict solution of the HJB equation 
if73j) ifveD (C ) and {73J) holds on [0,T] x R". 

Definition 4.6 A function v G C 0,1 ([0,T] x R n ) is a strong solution of the HJB equation 
IJ.'jj) if, setting h (t, x) = —H° (t, x, d x v (t, x)), v is a strong solution of the backward Cauchy 
problem 

Cqv (t, x) = h (t, x) , v (T, x) = (j) (x) , 

in the sense of Definition \4-2\ 

We now present our verification theorem in the extended version, as announced in Section 
[5] We need the following assumption. 

Hypothesis 4.7 There exists a function v G C 0,1 ([0,T] x R") which is a strong solution 
of the HJB equation \13\) in the sense of Definition \4 ■ b\ and is polynomially growing with its 
space derivative in the variable x. Moreover: 

(i) either we can choose the approximating sequence (v n ) of Definition \4~o\ so that for 
every < t < s < T and for every admissible control z G Z a d it) 



lim / {d x v n (r,y(r))-d x v(r,y(r)),F 1 (r,y(r),z(r)))dr = 0, u.c.p., 

n — >+oo J * 

(ii) or the function 

(t, x, z) — > B^ 1 (t, x) i*i (t, x, z) , 

where B^ 1 stands for the pseudo-inverse of B, is well defined and bounded on [0,T] x 
R" x U. 

Remark 4.8 All the results below still hold true with suitable modifications if we assume 
that: 



it) 



• the strong solution v belongs to C° ([0,T] x R") n C ' 1 ([e,T] x R") for every small 
e > 0; 

• for some (3 G (0, 1) the map (t, x) ->■ t^u (t, x) belongs to C° ([0, T] x R"). 

XTie proof of Theorem \4-iA in this case is a straightforward generalization of the one 
presented here: we do not give it here to avoid technicalities since here we deal with a model 
problem. In Section^ we will treat a case with such difficulty. See also Remark 4-10 of j 
on this. 



We give now here the precise statement of Theorem 12. 71 

Theorem 4.9 Assume that Hypotheses QH1 HOI \<2.4\ and \4.1\ hold. Let E% v , H° be as in 

0)-E|). Let v G C ' 1 ([0,T] x R") be a strong solution of EJ$ and fix (t,x) G [0,T] x R" 
which is polynomially growing with its space derivative in the variable x. Then 

(i) v< V on [0,T] xl". 

(ii) If z is an admissible control at {t, x) that satisfies (setting y (s) = y (s; t, X, z)) 

H° (s, y (s) , d x v (s, y (s))) = E% v (s, y (s) , d x v (s, y (s)) ; z (s)) , (16) 

for a.e. s G [£, T], P— almost surely, then z is optimal at (t, x) and v (t, x) = V (t, x) . 

The proof of this theorem follows by the following fundamental identity that we state as 
a lemma. 

Lemma 4.10 Assume that Hvvotheses Wl\ W3. \H\ and\Jl\ hold. Letv € C° 4 ([0,T] x R") 
be a strong solution of iji3j) . Then, for every (t,x) G [0, T] x R" and z G Z a( i (t) such that 
J (t, x; z) < +oo the following identity holds 

J (t, x; z) — v (t,x) 

+eJ [-H (s, y(s), d x v (s, y (*))) + H° cv (s, y(s),d x v (s, y (*)) ; z(s))] ds (17) 

where y(s) — y (s;t,x, z) is the solution of 0) associated with the control z. 

Proof. For the sake of completeness we first show how the proof goes in the case 
when v G C 1 ' 2 ([0, T] x R n ) and is a strict solution of (|T3|> . We use Ito's formula applied to 
the function v (t, x) by obtaining, for every (t, x) G [0, T] xK", 

v(T,y(T))-v(t,y(t)) = J (d x v(s,y(s)),B(s,y(s))dW(s)) 

+ J (F ( S ,y(s)),d x v(s,y(s)))ds (18) 

+ / (Fi (s, y(s),z (s)) , d x v (s, y (s))) ds 

+ / d s v(s,y(s))ds 
1 f T 

+2 J Tr \ B * ( s , y (*)) d ** v ( s - y (*)) B ( s - 2/ ( s ))l ds > 
ii 



(which is exactly the decomposition of Proposition 2.4 of [221 ) so j by taking expectation 
of both sides (this is finite since we have the polynomial growth assumption) we get the 
so-called Dynkin formula 

E v (T, y (T)) =Ev (t, y (i))+E / C v (a, y (a)) ds+E f (F 1 (a, y{s),z (a)) , d x v (a, y («))> da. 



Now by l(T3|) we get, for every a > 0, 

C v (a, y (a)) = -H° (a, y (a) , d x v (a, y (s))) , 

which yields 

Ev(T,y(T))=Ev(t,y(t)) 

+E / [-H (a, y (a) , d x v (a, y (a))) + (F, (a, y(s),z (a)) , d x v (a, y (a)))] da. 



Now since we assumed that J (t,x;z) < +cxd and since Hypothesis 12.21 implies J (t, x;z) > 
— oo (this also follows observing that the last line is not smaller than — J (t, x; z) and that 
the first line is finite by the polynomial growth of v), we add J (t, x; z) to both terms, use 
that v (T, y (T)) = 4>{y (T)) and that y{t)=x to get 

J (£, x; z) = v (t, x) + E / -if (s, y (s) , 9 X « (s, y (s))) ds 



+E / [(F! (s, i/ (s) , z (a)) , 9 x v (*, y (a))) +l(a,y (a) , z («))] ds 



which is the claim (|17|l . 

We now show how the proof goes when v € C 0,1 ([0, T] x R"). The difference is due to 
the fact that the term 

1 f T 
d s v (a, y (a)) ds + - / Tr [B* (s, y (a)) a M « (s, y (a)) B (a, y (a))] da 
t z Jt 

in the third and fourth line of equation l|18|) is not well defined now. However by Hypothesis 
14.71 we know that v is a strong solution (in the sense of Definition 14.211 of 

Cqv (t, x) = ho (t, x) , v (T, x) — if) (x) , 

where we have set ho (t, x) = — H Q (t, x, d x v (t, x)). Then applying now Theorem 14.31 for the 
operator Cq (setting b — Fq, <j — B and b\ = i*o + F\) we get that 

v (T, y (T)) = v(t,y (t)) + J (d x v (a, y (a)) ,£(*,» (a)) dW (a)) 

+ / [~H° (a, y (a) , 8 x v (a, y (a))) + (F, (a, y («) ,2 (a)) , X « (a, y (a))}] da. 

Now, taking expectation, adding and subtracting J (t, x; z) (which is a.s. finite using the 
same argument for the smooth case), using that v (T, y (T)) = (j> (y (T)) and that y (£) = x 
we get 



Z(a,y(a),z(a))da + </>(2/Cn) 



w (t, x) 
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+E j [-H (a, y (s) , d x v (a, y(s))) + H° cv (a, y (a) , d x v (a, y(a));z («))] ds 
and the claim again follows. I 

Proof of TheoremlHU By the definition of H° and H% v , for every (t, x) G [0, T] x R n , 
z G Z a d it) and for a.e. s G 7t, the following inequality holds P-almost surely (setting 
2/0) = 3/ (s; t, a:, z)) 

-H° (s, y (a) , d x v (a, y (a))) + H° cv (a, y (s) , d x v (a, y (a)) ; z (a)) > 0, (19) 

and then by the fundamental identity l|17|l it follows v (t, x) < J (t, x; z) for every admissible 
z. This gives v < V and so part (i) of the Theorem. 

Now consider an admissible control z such that, for a.e. s 6 [i, T], P - a.s. 

H° (a, y (a) , d x v (a, y (a))) - H° cv {a, y (a) , d x v (a, y (a)) ;z(a)). (20) 

By the fundamental identity (|17|) we have 

V (t, x) = J (t, x; z) , 

which implies optimality and v (£, x) = V (t, x). I 



Remark 4.11 To get the methodological novelty in the above proof it is crucial to note the 
following. Since we know that u is the limit of classical solutions, the first idea to approach 
the above proof (and that has been used e.g. in the papers flfft \21f). is probably to prove 
the fundamental identity for the approximating solutions u n and then pass to the limit for 
n — > oo. However if one tries to do this one needs to use the uniform convergence of the 
space derivatives d x u n to d x u. This is not needed with our method, see also on this Remark 

E3 ■ 

Let us go now to the proof of Verification Theorem 12. 81 for the infinite horizon case. Set 
T = +oo, assume that Fq, F\ and B do not depend on time and that I (t, x, z) = e xt l\ (x, z) 
(A > 0), 4> = 0; then we can set t = (since the dependence of the value function V on t 
becomes trivial in this case) and the HJB equation for Vq becomes an elliptic PDE. 

We define the operator 

U : D (L ) C C° (R n ) — ► C° (R n ) , D (L ) = C 2 (M n ) , 

L u (») = (F (x) , d x u (»)> + -Tr [B* (x) d xx u (x) B (x)} 

and we rewrite the HJB equation (|1()|> as 

Xu(x)+L u(x) +H 1 (x,d x u(x)) = 0, x £ R". (21) 

where H 1 is defined in l|ll|l . We now define strong solutions for such equations. Consider 
first, for h G C° (R™) the inhomogenous elliptic problem 

Xu(x)+L u(x) + h(x) = 0, Viel". (22) 

Definition 4.12 We say that u is a strict solution to the elliptic problem \2ty if u G D (Lq) 
and 12%\) holds. 
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Definition 4.13 We say that u is a strong solution to the elliptic problem \22\) if there 
exists a sequence (u n ) C D (Lq) and a sequence (h n ) C C'° (R"), such that 

1. For every n E N u n is a strict solution of the problem 

Xu n (x) — Lqu„ (x) = h n (x) , Vx E R™. 

2. The following limits hold 

u n — >u inC° (R™) , 
hn — >hinC" (M™) . 



For functions u E C 1 (R n ) that are strong solutions of the Cauchy problem 121' I) the 
result of Theorem 14 . 31 still holds: it is indeed a simpler case. Now we go to strong solutions 
of the HJB equation (|5T]l . 

Definition 4.14 A function v E C 1 (R™) is a strong solution of the HJB equation \21\i if 
setting ho (x) — —H 1 (x,d x v (x)), v is a strong solution of the linear problem 

Xv (x) + L v (x) — ho (x) , 

in the sense of Definition \4 ■ 1$\ 

Assume the following. 

Hypothesis 4.15 There exists a function v E C 1 (K™) which is a strong solution of the 
HJB equation \2ty in the sense of Definition ^. il\ Moreover: 

(i) either we can choose the approximating sequence (v n ) of Definition \J~1~%\ so that for 
every s > and for every admissible control z E Z a( i (0) 

lim / (d x v n (y(r))-d x v(y(r)),Fi(y(r),z(r)))dr = 0, u.c.p., 

(ii) or the function 

(x, z) — > B^ 1 (x) F\ (x, z) , 

where B^ 1 stands for the pseudo-inverse of B, is well defined and bounded on K" x U . 

The precise statement of the verification theorem is the following. 

Theorem 4.16 Assume that Hvvotheses \2.1\ and \4-l5\ hold. Assume also that l\ is con- 
tinuous and bounded and that Hypothesis \2.J\ hold true with H 1 in place of H . If v is a 
bounded strong solution of the HJB equation $21\) and v E C 1 (R™) then v < Vq on R™. 
Morerover if z is an admissible control at (0, x) that satisfies (setting y{s) = y (s;0,x, z)) 

H 1 (y (a) , d x v (y (s))) = H l cv (y (s) , d x v (y (s)) ; z (s)) 

for a.e. s E [0, +oo), P— almost surely, then z is optimal and v (x) = Vq (x) . 

Proof. The proof goes along the same lines as the finite horizon case. First we observe 
that the boundedness of the datum l\ implies that also Vq is bounded and that for every x E 
R n , z E Z a d (0) the functional J (0, a;; z) is finite. Then we prove the following fundamental 
identity 

/■+OO 

J (0, x;z)=v(x)+E e- Xs [-H 1 (y(s), 8 x v (y («))) + E x cv (y(s), d x v (y (s)) ; z(s))] ds. 
Jo 
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To do it we apply Theorem 14.31 for the operator Co = dt + Lq (so b = Fq , a = B and 
61 = Fq + Fi) but taking t — and replacing v (s, x) by e~ Xs v (x) for s > 0. Observe first 
that, if v G C 2 (M n ) then the function to (i, *) = e~ xt v (x) solves, for t > 0, x G R" the 
equation 

w t (t, x) + L Q w (t, x) = -e- xt H x (x, d x v (x)) . (23) 

Moreover if v G C 1 (PJ 1 ) is a strong solution of (|21[1 . also to (i, cc) is a strong solution of (|23|l 
and satisfies the assumptions of Theorem 14.31 So we get for every T\ > 

e- ATl ^(2/(T 1 )) = W ( 2 /(0))+ / 1 e- As (5 xU (y(s)),B( 2 /( S ))dW( S )> 

Jo 

Ti 





v [-Jf x (s, ]/ (s) , d x v (s, y (a))) + (F t (s, y(s),z (*)) , 9 xW («, v («))>] da. 



Now, taking expectation, adding and subtracting E J x e A5 7i (y (s) , z (s)) ds (which is a.s. 
finite by the boundedness of l\) and using that y (0) = x we get 



E 



1 e- As Z! (y ( S ) , z ( S )) ^ + e- AT M2/ (TO) 





Ti 

As 



= v(x)+E e- Xs [-H 1 (y( S ),d x v(y( S )))+H 1 cv (y(s),d x v(y( S ));z( S ))]d S 

Jo 

and the claim again follows taking the limit for T\ — > +00 and using the boundedness of u. 

The rest of the proof is exactly the same as in the finite horizon case. I 

Remark 4.17 It must be noted that the assumption on the boundedness of l\ and v is made 
for simplicity of exposition. What is really needed in this infinite horizon case is that J is well 
defined for each z G Z a d (0) and that limj'i^+oo e~ XTl v (y (Ti)) = for every z G Z a d (0). 
This allows to pass to the limit in the latter formula. In some cases this can be checked 
directly, in other cases much weaker conditions can be imposed (e.g. sublinearity of l\ and 
estimates on the solution y as T\ — > +00J. I 



5 Necessary Conditions and Optimal Feedback Controls 

Here we want simply to show that under additional assumptions (mainly about existence 
and/or uniqueness of the maximum of the Hamiltonian and of the solution of the closed 
loop equation) we can get necessary conditions and optimal feedback controls. This is 
a consequence of the verification Theorems 14.91 and 14.161 which has some importance for 
applications. For the sake of brevity we give the results only for the finite horizon case 
observing that completely analogous results hold true for the infinite horizon case. We start 
by making a remark about the so-called "weak formulation" of a stochastic control problem. 

Remark 5.1 The setting introduced in Section\^ corresponds to the so-called "strong for- 
mulation" of a stochastic control problem. For certain purposes (namely the existence of 
optimal feedbacks) it is convenient to consider the "weak formulation" , letting the stochastic 
basis vary. In such formulation one considers as the set Z a d (t) of admissible controls as the 
set of 5-tuples (fi, J 7 , P, W, z) such that 

• (fi, F, P) is a complete filtered probability space with the filtration T satisfying the usual 
conditions, 
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• W is an E valued m- dimensional Brownian motion on [t,T], 

• the process z is measurable, (J-'s)- adapted and a.s. locally integrable. 

We will use the notation (f2, J-, P, W, z) G Z a d (t) and we will call this control strategies 
weakly admissible (weakly optimal when they are optimal). When no ambiguity arises we 
will leave aside the probability space and the white noise (regarding it as fixed) and simply 
consider z G Z ac i (t). We will specify when the weak concept of admissible control is needed. 
The same will be done in Sections^ and\^ One can see e.g. ]39l p.64], ]1& pp.141, 160] 
for comments on these formulations. I 

We start recalling a case when the sufficient condition of the verification Theorem 14.91 
becomes also necessary. 

Proposition 5.2 Assume that the hypotheses of Theorem \4.y\ hold true. We also assume 
that the value function V is a strong solution of the HJB equation. Then an admissible 
control z G Z a d (t) is optimal at (t, x) G [0, T] x R™ if and only if fi6j) holds with V in place 
ofv. 

Proof. The fundamental identity in this case gives 

J(t,x;z) =V(t,x) 

+E J [H° (a, y (a) , d x V (a, y(s)))- H° cv (a, y (a) , d x V (a, y(s));z (a))] da 
and this immediately gives the claim. I 

Remark 5.3 One caae where the above Provosition \5.S\ can be applied is when: 

• it is known that V is the unique viscosity solution of the HJB equation; 

• it is known that strong solutions are also viscosity solutions. 

This happens e.g. in the example of Section^ | 

We now pass to the existence of feedbacks. First we recall the definition of (weakly) 
admissible feedback map. 

Definition 5.4 A measurable map G : [0, T] x l n i — ► U is a (weakly) admissible feedback 
map if for every (t,x) G [0, T] x H." the closed loop equation (s G [^T 1 ]^ 

y{s)=x + j F 1 (r,y(r),G(r,y(r)))dr + J F (r,y (r)) dr + J B {r,y (r))dW(r), (24) 

admits a (weak) solution y = ya,t.x for every (t, x) G [0, T] x W 1 and the control strategy 
z G,t,x(s) — G (s,yc,t.x (s)) is (weakly) admissible. 

If we have a (weakly) admissible feedback map G then the control strategy zc,t,x (s) = 
G (a, yc,t,x (s)) is by definition (weakly) admissible and so, given G, we have, for every 
(t, x) G [0,T] x R™ an admissible (weak) couple (zc,t,x,yG,t,x)- Our goal is to find an 
admissible feedback map G such that, for every (t,x) G [0, T] x R", the strategy zc,t,x is 
(weakly) optimal. Such G will be called a (weakly) optimal feedback map. 
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Proposition 5.5 Assume that the Hypotheses of Theorem ^. y\ hold true. Assume also that: 

(i) the maximum of the Hamiltonian 01) exists and it is possible to define a measurable 
map 

G : [0, T]xl"x K n < — > U 

such that Go (t, x,p) £ argmin z Hqv (t, x,p; z) for every (£, x,p) £ [0, T] x R™ x R"; 
(ii) the map G (£, x) = Go (t, x, d x v (£, x)) is a (weakly) admissible feedback map. 

Then for every (t,x) £ [0,T] x R™, zc,t,x is a (weakly) optimal control, yc,t,x is the 
corresponding (weakly) optimal state and v = V . 

Finally i/argmin^jj Hqv (t> x i v x {t, x); z) is always a singleton and the closed loop equa- 
tion \2$ admits a unique (weak) solution, then the optimal control is (weakly) unique. 

Proof. This is an obvious consequence of the verification Theorem 14.91 I 

Remark 5.6 We observe that if we have existence of the (weak) solution of the closed loop 
equation \2J$ for every strong solution of the HJB equation f!3p. then we automatically have 
that the strong solution is unique. I 

Remark 5.7 We finally observe that Propositions \£^ and 15.51 holds also in the infinite 
horizon case with obvious changes. We will use the infinite horizon version of Proposition 
15.51 in the study of the one dimensional example in Section^ I 



6 Application 1: a class of exit time control problems 
with non degenerate diffusion 

Here we show that our technique for proving verification theorems works for a family of 
stochastic optimal control problems with exit time and nondegenerate diffusion where the 
solutions of the associated HJB equation are not known to be G 1 ' 2 . 

6.1 The problem 

Let first M. n be the state space, M. n be the space of noises and U (a given subset of a Polish 
space) be the control space. Let T be a fixed finite horizon, (CI, T, P) be a given stochastic 
basis (where T stands for a given filtration (J r s ) se i T , satisfying the usual conditions), W 
be a cylindrical Brownian motion with values in 1" adapted to (•^s) se foTl' Consider a 
stochastic controlled system in W 1 with fixed finite horizon T and initial time t £ [0, T) 
governed by the state equation 

dy (s) = [F (s, y (s)) + F 1 (s, y (s) , z (s))] ds + B (y (s)) dW(s), s £ [t, T] 

y(t) = x. 

We consider an open bounded domain O C W 1 with uniformly G 2 boundary (see e.g. |H()I 
pp. 2-3] for the definition). The initial datum x belongs to O and we call tq the first exit 
time of the process y from this open set, i.e. 

to (u) = inf {s >t :y (s; w) £ O c } . 

Moreover F :[0,T]xO-> R n , Fi : [0, T] x 6 x U -» W\ B : [0, T] x 6 -> £ (R n ; W l ), [6 
stands for the closure of the set O) satisfy 
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Hypothesis 6.1 Fq and F± are continuous and bounded. 

Hypothesis 6.2 B is time independent, uniformly continuous and bounded and nondegen- 
erate i.e. there exists \q > such that 

n 

K^U^ E [b{x)b t {x)] ij U 3 >\ \U V((,x)e[0,r]xO, V£eK". 

Remark 6.3 First note that the above Hypothesis \b.2\ implies that B is invertible with 
bounded inverse, so we fall in the Hypothesis \4- r /[ (ii)- Moreover it is possible to extend 
our results to the case when B is time dependent, provided suitable regularity conditions 
are satisfied. Such conditions are needed to apply the results of semigroup theory in the 
subsection below and substantially require that the domain of the operator 

L(t)u = ^Tr[B*{t,x)B(t,x)d xx u] 

be constant and the coefficient B be Holder continuous in time, see on this OV and \3(\ 
Section 3.1.2]. We leave aside these assumptions to keep a more simplified setting where, 
anyway, C ' regularity does not hold in general. I 



About the control strategy z : [t, T] x Q —> U we assume that it belongs to a given set 
Z a d (t) of stochastic processes defined on [t, T] x $7 with values in a fixed Polish space U. 
More precisely we will assume the following. 

Hypothesis 6.4 The control space U is a Polish space, while Z a d (t) is the space of all 
measurable processes z : [t, T] x 51 — > U adapted to the filtration J- . 

Remark 6.5 The above setting corresponds to the so-called "strong formulation" of a stochas- 
tic control problem. In this problem, since the state equation admits only weak solutions in 
general (see next proposition) we need to consider the "weak formulation" , letting the stochas- 
tic basis to vary, see Remark \5.1\ To avoid heavy notation we will keep the same symbols as 
in the strong formulation, as we did in Section^ I 



The following Proposition can be proved in the same way as in [37j , even if the drift is 
random. In fact Theorem 6.4.3 therein makes use of Girsanov transformation to eliminate 
the drift; the same can be done here. Then we can apply Corollary 6.4.4 and Theorem 7.2.1 
of 127]. 

Proposition 6.6 Assume that Huvotheses \6. l[\f>. 'A hold. Then, for all z G Z a( i (t), equation 
\2tyl has a weak solution 

y(-,t,x,z)eC r(t,T;X). 

This solution is unique in the sense of probability law. 

Remark 6.7 In the case of dimension 1, no continuity on the diffusion coefficients is re- 
quired, see \31\ Exercise 7.3.3 at page 192. I 
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We now consider the following stochastic optimal control problem with exit time. Mini- 
mize the cost functional 



J(t,x;z) = E 



to AT 

l(s,y(s;t,x,z),z(s))ds (26) 

t 



+ i {t <t}^ {to,V {ro;t, x, z)) + I{T >T}<i>{y {T; t, x, z))] , 

over all controls z G Z a d(t). Here y(-;t,x,z) is the solution of the equation IJ2"B|) and we 
assume that /, ip, <fi satisfy 

Hypothesis 6.8 I G C° ([0,T] x 6), (j> G C° {&) i/> G C^' 1+l3 ([0,T] x dO) (for some 
13 >Q), ip(T,x) = 4>{x) on dO. 

Remark 6.9 The above hypothesis is needed to apply the results of semigroup theory in the 
subsection below. In particular, under Hypothesis \6.8l the operator 

Lu = -Ti[B*(x)B(x)d xx u] 

(with Dirichlet boundary conditions) generates an analytic semigroup and the associated 
boundary value problem in O is well posed (see ]3(A Ch.5]). I 



The value function of this problem is defined as 

V{t, x) = inf {J (t, x\z) : z£ Z ad (*)} (27) 

and a control z* G Z ac i(t) and such that V(t,x) — J(t,x;z*) is said to be optimal with 
respect to the initial time and state (t,x). The corresponding HJB equation is 

' v (t, x) + |Tr [B (x) d xx v(t, x)B* (x)} = (F (t, x) , d x v (t, x)) + H° (t, x, d x v (t, x)) , 

t€[0,T], xeO, 
v(T,x) = <f>(x), xeO, 
v(t,x) = i> (t, x), t G [0, T] , x G dO, 



where 



(28) 
H° (t,x,p)= inf H% v (t,x,p;z), (29) 



with 

Hc V (t,x,p;z) = (F 1 (t,x,z),p) +l(t,x,z), 

being the current value Hamiltonian. 



Proposition 6.10 Under Hvvotheses \6. 1\\6. &\6.4\ F^"%1 the Hamiltonian H° (t,x,p) defined 
in 129(1 is continuous in [0, T] x O x W 1 . Moreover there exists a constant C > such that 

\H (t,x lP )-H (t,x iq )\ < C\p-q\ 

\H°(t,x,p)\ < C(l + |p|) (30) 

Proof. It is enough to apply the definition of the Hamiltonian and use the continuity 
and the boundedness of F\ and I. I 
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6.2 Strong solutions of the HJB equation 

The HJB equation l|28[) above is a semilinear parabolic equation with continuous coefficients. 
Since the second order term is nondegenerate (Tfypothcsis 16 . 2[) one expects interior regularity 
results for the solution even if the boundary data are merely continuous, as it is under our 
assumptions. In particular the following result, taken from 9 , Theorems 9.1 and 9.2, apply. 

Theorem 6.11 Under Hvvotheses \b\ 1\ \(j.'A \b'-4[ I6'.£l there exists a viscosity solution u of 
113) and u belongs to the space C^f' 1+a ([0, T) x O) D C° ([0, T] x 6) for some a e (0, 1). 

We note also that it is not known if the solution is classical in the interior. This is known, 
using theorems of [5], if the data are supposed to be Holder continuous in (t, x), which we 
do not assume here. Moreover even in the linear case, if the coefficients are only continuous, 
solutions are only W 2 ' p , so they are also C l,a . An example in this direction is in ^J Ch.4]. 

This means that we are exactly in the case where it makes sense to apply our technique 
to prove verification theorems: no classical solution but solutions (in a generalized sense) 
with at least C ' 1 regularity. Once this is clear, we need to check if our solutions (that exists 
at least in the viscosity sense thanks to the above Theorem 16. 11 ll are also strong solutions in 
the sense of Definition 14.61 fsuitablv modified to take care of the boundary datum ip). Such 
a result is not available in the literature in this form but it can be easily deduced using the 
results on analytic semigroups contained e.g. in [jJOJ. We explain here below how this can 
be done in the case when if; = (the general case can be treated with the ideas explained 
in PHI Section 5.1.2]). In this case the operator 

L : D(L)CC° (6) -» C° (6) , 
D(L) = (1,6 0^(0): V ,L V EC°(d), V \oo = o] , 

(Lr))(x) = l -Ti[B{x)d xx i 1 {x)B*(x)] 1 

generates an analytic semigroup {e tL ,t > 0}, [301 P-97, Corollary 3.1.21]. Moreover, given 
any initial datum cf) E C° {&) and any function H° continuous in [0, T] x O x R" and 
satisfying (|30|l we can apply a modification of the argument used to prove Proposition 7.3.4 
in (301 p. 281] to get existence and uniqueness of a solution u of the integral equation 

u(t,x) = U T -^ L (f>) (x) (31) 

[e(T-s)L ^ Q (Sj . } ( dxU (S; . )} + ^o (S; . ; dxU (Sj . }) ] ) {x) dSj 

which can be considered as an integral form of the PDE l|28|l when xp = (in the general 
case one needs to lift the function ip into the equation obtaining an extra term in Ij31|l . see 
on this [3D 1 , Remark 5.1.14, p. 195]) written using the variation of constants formula. More 
precisely we have the following definitions and results. 

Definition 6.12 We say that a function w belongs to the space S 1 ' ([0, T] x O) if w S 
C° ([0,T] x O), w is Frechet differ entiable in x G O, d x w G C° ([Q,T - e] x 6) for every 
e G (0,T), and 

sup \(T — t) a d x w (t, x)\ < +oo. 

(t,a:)e[0,T)xO 

Definition 6.13 A function u £ S 1 '' ([0,T] x O) that satisfies the integral equation |32J) 
for every (t,x) £ [0,T] x O is called a mild solution of the HJB equation $28\) . 



20 



Theorem 6.14 Assume that B E C° (O) , _F E C° ([0,T] x O), 4> E C° (O) and %/> = 0. 
Moreover let H° be continuous in [0,T] x O x ffi™ and satisfies h3U\) . Then there exists a 
unique mild solution u E S 1 '^ ([0, T] x O)of the HJB equation \2fy) . 

Proof. The proof is a standard application of the contraction mapping principle, see 
|3HI Proposition 7.3.4] and also piT%]. I 

Now we want to show that such a mild solution is a strong solution in the sense that it 
can be seen as the limit of smooth solutions. 

We rewrite here the definition of strong solution for our case since it is slightly weaker 
than the one used in SectionQ]due to the possible singularity of the spatial gradient at t = 0, 
see Remark [ 



Definition 6.15 Let(j)E C° (O) andip E C°([0,T] x dO). Afunctionu E E 1 ^ ([0,T] x O) 
is a strong solution of the equation \2ty) if it satisfies the boundary and final conditions 

u (T, x) — 4> (%) , x E O, 

u (t, x)=ip(t,x), tE [0, T] , s E dO, 

and if for every e > 0, it is a strong solution (in the sense of Definition ^- 3j) of the linear 
parabolic problem 

w t +Lw = h, tE [0, T], x E O, 
w(T — e,x) = u(T — e,x), x E O, 

where h = [(Fq (-,■) ,d x u (■,■))+ H° (-,-,d x u (■,■))], i.e. if, for every e > there exist 
sequences u £ n , h E n , (j) e n such that 

1. for every n u e n is a strict solution (i.e. it belong to C 1 ' 2 ([0,T — e] x O) and satisfy 
the equalities on [0, T — s] x O) of the approximating problem 

v t + Lv = h e n , v{T-e,x)=4> e n {x); 

2. u e n converges to u uniformly in [0, T — e] x O, as n — » +oo; 

3. h e n converges to h uniformly in [0, T — e] x O, as n — » +cxd; 

4- 4>n — u(T — e, •) converges to zero uniformly in O, as n — > +oo. 

Given the above Definition IB . 1 5l we can apply directly Proposition 4.1.8 (see also Theorem 
5.1.11) of [HO] to get the following. 

Theorem 6.16 Under the same assumptions of Theorem \b.lI\ the mild solution of H281) is 
also strong. 

Proof. It is enough to apply Proposition 4.1.8 and Theorem 5.1.11 of [3U] to the 
nonhomogenous linear parabolic problem 

vt + Lv = h, v (T, x) =u(T — e, x) , 

where we set 

h (t, x) = (F (t, x) , d x u (t, x)) + H° (t, x, 8 x u (t, x)) . 

In fact for every e > such function h belongs to the space C° ([0,T — e] ;C° (O)) = 
C°([0,T-e]x<5). ■ 
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Remark 6.17 In the case when if is not the above results still hold true using the same 
techniques shown in ]HUf . Theorem 5.1.16, and 5.1.17. We do not do it here for simplicity 
of exposition. ■ 



Remark 6.18 Suppose that the operator L can be written in divergence form and that the 
coefficients are only Borel measurable and not necessarily continuous; suppose moreover that 
the diffusion coefficients are lower and upper bounded by a constant. Then, the semigroup has 
a density with respect to the Lebesgue measure and it fulfills the classical Aronson estimates, 
see for instance (TJ \8ftf . 

Fukushima - Dirichlet decomposition for mild or weak solutions to equations of type i28\) 
were treated by "2, 26, 351. Using such kind of results or possible generalizations in the spirit 
of ]22f . one could establish verification theorems related to optimal control problems even in 
that framework. I 



6.3 The verification theorem 

Now we prove a verification theorem for the optimal control problem above Ij25(l - (|26|l . The 
proof is a modification of the proof given in Section 0J The main differences are due to 

1. a singularity of the first derivative and so a different definition of strong solution; 

2. the constraint on the set O and so the presence of boundary data in x (exit time). 

The statement of the verification theorem in this case is the following 

Theorem 6.19 Assume that Huvotheses PHI El WH T6~R hold. Let v e S 1 ^ ([0,T] x 6) 
be a strong solution of \2ty) . Then 

(i) v< V on [0,T] x 6. 

(ii) If z is an admissible control at (t,x) € [0,r] x O that satisfies (setting y(s) = 
y(s;t,x,z)), F-a.s., 

H° (s, y (s) , d x v (s, y (s))) = H% v (s, y (s) , d x v (s, y (s)) ; z (a)) , 

for a.e. s € [t, T A to], then z is optimal at (t, x) and v (t, x) — V (t, x) . 

The proof of this theorem follows as usual by the following fundamental identity that we 
state as a lemma. 

Lemma 6.20 Assume that Huvotheses \6~l\\6~2\ \lTj\ E3 hold. Let v e S 1 ^ ([0,T] x O) be 
a strong solution of \2i^ . Then, for every (t,x) G [0, T] x O and z £ Z a d (t) the following 
identity holds 

J (t, x; z) — v (t,x) 



pr /\T 

+E / [-H (s, y(s), d x v (s, y (a))) + H° cv (a, y(s), d x v (a, y (a)) ; z{s))] ds, (32) 

where y(s) = y(s;t,x,z) is the solution of \25\) associated with the control z. 
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Proof. Since v G S 1 '^ ([0, T] x O) then for every e > we have u e C ' 1 ([0, T - s] X O) 
and v is a strong solution of the problem 

wt + Lw = ft, te[0,T], x e O, 
w(T — e,x) — v (T — e,x), x<eO, 

where h = [(F (•, •) , ^w (•, •)} + H (•, •, 5 x i; (•, ■))], (see Definition EEJ) . 

Applying now Theorem 14. 31 and Remark 4.9 of [22] for the operator £q = dt + L (so now 
b = 0, er = B and 6i = Fo + Ft), we get that 

« (ro A (T - e) , y (to A (T - e))) = v (t, y (t)) 



roA(T-e) 



(d x v(s,y(s)),B(y( s ))dW(s)) 



ToA(T-e) 



fi r °(s,y(a),9 x v(s,j/(s)))ds 



roA(T-e) 



(Fi (s, y(s),z (s)) , ^w (s, y (s))) ds. 



r r A(T-e) 



Adding and subtracting J ° I (s,y (s) , z (s)) ds (which is always finite since / is bounded), 



using that y{t) = x and taking the expectation we get 

r T A(T-e) 



E 



I (s, y(s),z (s)) ds + v {t A (T - e) , y (to A (T - e))) 



= v (t,x) 



/■ToA(T-e) 

+EJ [-H°(s,y(s),d x v(s 1 y(s))) + H cv (s 1 y(s),d x v(s,y(s));z(s))]ds. 

Let now e — > + . By continuity of u (and since by definition the strong solution satisfies the 
boundary and final condition) we get, 

Ev (t A (T - e) , y (t A (T - e))) -> E [/ {ro > T }0 (2/ (T)) + I {to <t}4> (t<d,V (t©;*, ar, «))] . 
Moreover, by the boundedness of Z we get 

/•toA(T-e) /-toAT 

E/ l(s,y(s),z(s))ds->E l(s,y(s),z(s))ds, 

Jt Jt 

Finally we recall that, by the definition of the Hamiltonian (|29|l and thanks to the fact that 
v E £ 1: ^ ([0, T] x O), for a suitable C > we have for every s <E [0, T) 

E [-#° (,, y (s) , S^ (s, y («))) + ff° y (s, y (s) , a t; (s, y («)) ; z (*))] ds 

< C (l+E\d x v(s,y(s))\)<C 1 (T-syK 

which allows us to apply the dominated convergence theorem obtaining 

-T A(T-e) 



E 



[-F (s, y (s) , a^ (s, y (s))) + ff£ y (s, y (s) , d^ (s, y (s)) ; z (a))] ds 



ro AT 



E / [-#° (s, y (s) , 9 x v (s, y (s))) + H° cv (s, y (s) , d x v (s, y (s)) ; z (s))] ds. 
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All the above yields then the claim 13211 . | 

Proof of Theorem 16.191 By the definition of H° and Hq V , for every z G Z a d (£), 
x G O the following inequality holds P-almost surely (setting y (s) — y (s; t, x, z)) 

H° (s, y (s) , d x v (s, y (s))) - H% v (s, y (s) , d x v (s, y (s)) ; z (s)) > 0, for a.e.s (= [t, T A r G ] . 

(33) 
The rest of the proof follows exactly as the one of Theorem 14.91 I 



Remark 6.21 Arguing as in Subsection^ it is possible to prove a necessary condition and 
the existence of weakly optimal feedbacks. I 



7 Application 2: a class of problems with degenerate 
diffusion 

Now we consider our model problem focusing the case where the HJB equation admits 
solutions that are C 0,1 but not C 1,2 . This may occur for instance in the case when the 
matrix BB* is degenerate in the sense that there exists at least one point x € K™ such that 
BB* is not invertible. 

Indeed such degeneracy means that the HJB equations i|13|) and (|21|l are degenerate 
(parabolic or elliptic). For such equations a general regularity theory like the one available 
in the uniformly parabolic or elliptic case is not known. So it is difficult to obtain existence 
of C 1,2 (C 2 ) solutions in this case. There is more hope to obtain solutions with regularity 
C ' 1 (C 1 ) or at least continuous with existence of the space derivatives in the weak sense 
(this case is not covered in the present paper but is the subject of our current research). This 
means that, on one hand, this kind of problems is a source of possible sharp applications of 
our verification Theorems 14.91 and 14 . 1 61 On the other hand in this case it is difficult to have 
Hypothesis I4.7l (ii) to be satisfied. So one needs to have convergence of the derivatives in 
the sense of Hypothesis 14.71 - (i). which may be not easy to check. We give an example when 
this can be done in the last part of this section. 

Remark 7.1 A possible methodology to prove existence of C 0,1 solutions of the parabolic 
degenerate equation |_?51) is the following (consider the case where all data are autonomous). 
Take the degenerate elliptic operator 

L lV (or) - (F (or) , d x n (x)} + ^Tr [B* (x) 8 xx n (x) B (x)} , 

defined in C 2 (R ra ). We can write the HJB equation 03J) as 

v t (t, x) + L x v (t, x) + H a (x, d x v (t, x)) = 0, 

v (T, x) = cj) (x) . 

At this point, if the operator L\ is "sufficiently good", e.g. if it generates a smoothing 
semigroup (see the papers "15, 20! on this) one can try to apply some perturbation arguments 
finding existence of a strong solutions. Also Remark \4-.ti\ applies. | 
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To illustrate the above situation we present a one dimensional example coming from 
optimal advertising models. For this reason the problem is taken with the sup, so also 
Hamiltonians are different from the rest of the paper. The proofs are only sketched for 
brevity. 

The state variable y is the goodwill (i.e. the number of people that are aware of the 
product), the control variable z is the investment in advertising and one maximizes the 
profit from selling the product on a given finite horizon; see on this e.g. |24l I.'i2| and the 
references therein. 

The state space is R, the control space is U = R + . The noise space is R and W is a 
standard 1-dimensional Brownian motion. Given an initial point x £ R and parameters 
a, f3 > 0, the state equation is 

dy (s) = [-ay (s) + z («)] ds + (3y (s) dW (s) 
y(t)=X. 

Given p > 0, c : R + — > R , continuous, increasing and convex and such that 

l im £ifl = +0O; (35) 

2^ + 00 Z 

ft : R — > K, continuous and strictly increasing, we maximize the profit functional 

J(t, x;z)=EU -c(z («)) ds + ft (j/ (T)) I , 

over all controls z G -Z a( j (t) where 

Z a d{t) = {z : % x fl ^ U, measurable, a.s. locally integrable, (^ r s )-adapted}. 

Here y = y (-;x, z) = y (■; t, x, z) is the strong solution of the equation l|34|) on [0, +oo). Here 
Hypotheses 12. II \2. 21 and I2T41 are clearly satisfied. The value function is 

V(t, x) = sup {J(t, x;z) : z e Z ad (*)} 

and the corresponding Hamilton- Jacobi-Bellman equation (HJB from now on) reads as fol- 
lows 

d t v (t, x) + \l3 2 x 2 d xx v{t 1 x) - {ax, d x v (t, x)) + H° (d x v (t, x))=0, 

t£[0,T], xeR, (36) 

V (T, x) = h{x), xeR, 

where the Hamiltonian H° is given by 

H° (p) = sup {zp-c(z)} = c*(p), 

z>Q 

(where c* is the Legendre transform of c) and it is always finite and continuous thanks to 
(|23. 

Remark 7.2 We stress the fact that, via the approach described in Remark \7.1[ we can 
prove, eventually restricting the assumptions on data, that there exists a strong solution of 
the HJB equation 1.%)) . In fact, the operator 

L\r) (x) = (ax, d x i] (x)) + -0 2 x 2 d xx ri (x) , 
25 



generates an analytic semigroup (see e.g. ]2(rf and the references therein) in the space C° 
with domain C 2 with suitable weights. This allows us, using a perturbation method similar to 
the one of [19l, to show, under some restriction on the data, the existence of a mild solution 
of HJB equation \3b)) . i.e. a solution of the integral equation 

v (t, ■) = eP-Q^h + f e (T - s)Ll ff° (d x v (s, -))ds 

belonging to the space C 0,1 with suitable weights. Such mild solution turns out to be strong 
in the sense of Definition \4-6\ easily: by simply taking suitable approximations of the initial 
datum h and of H° (d x v (s, •)), as it is done again in ]lty or in [SO], one finds solutions u n 
of approximating problems that converges to u. However it is not trivial to see if for such 
approximating sequence we also have the convergence required in Hypothesis \4 -T\ -(i). Below 
we see that it is true in a simple case. I 

Choosing data in a suitable way we can write explicitely a strong solution which satisfies 
our assumptions but is not C 2 in space. Let c(z) = z 1+v with 77 6 (0,1), and h (x) — 
\x\ '' sgn (x). Then the Hamiltonian is 



h°(p)=v m 



i+h 



with the maximum point 



(37) 



If we look for a solution of the form w (t, x) = / (t) ■ \x\ n we see that the function 

( a(t)\x\ 1+v , for a; > 0, 

v (i, x) = < 0, for x = 0, 

[ &(f)|2;| 1+ \ fora;<0, 

satisfies for every x ^ the HJB equation (|36(l above if a > and b < are, respectively, 
the solution of the Cauchy problem 



and, respectively, 

b'(t) = 



-l3 2 n{l + V )-a(l + V ) 



(t)-ria(ty^, a(T) = l, 



-f3 2 n (1 + rf) - a (1 + rf) 



b(t), b(T) = -l. 



Now the second Cauchy problem admits a unique global solution on [0, T], which is al- 
ways strictly negative. The first does the same under suitable conditions on the data 
(e.g. that \P 2 ri < a): assume from now on that this is the case. We can easily see that 
v E C 0,1 ([0, T] xR). Moreover we may prove, by approximating v by v n = v ■ 8 n (where 9 n 
is a suitable cut-off function such that 9 n = foi \x\ < — and 8 n = 1 for \x\ > — ) that v 
is a strong solution of (|36l) and that d x v n — > d x v uniformly on compact sets. This fact says 
that we have a strong solution to which standard verification do not apply but which falls 
into our assumptions. Indeed since the convergence of the derivative is uniform on compact 
sets also another technique can be applied (see Subsection 18. 2JI . 
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Remark that not only Theorem l4 . 9l but also Proposition l5 . 5l applies. Indeed the maximum 
of the Hamiltonian is reached in the unique point z given by %61\ . so we can set, with the 
notation of Proposition l5.5l 

G (t,x )P )=G (p)=(^—) . 



Recalling that 

( a (t) (1 + ij) \x\ v , forx>0, 

d x v it, x) = < 0, for x = 0, 

{ -bit) (l + rj) \x\ v , for x < 0, 

we have 

{a it)*> \x\ for x > 

for x = 

[—b(t)]'i \x\ for x < 

It is easy to check that G is an admissible feedback map (along with Dcfmition l5.4fl . In fact, 
setting, for x > 

z(s) = ais)~ \y(s)\, 

the closed loop equation is 

f dy(s)= -aey(s) + a(s)* \y(s)\ dt + /3y (s) dW (s) 
I y(t)=x. 

This equation always features existence and uniqueness of a strong solution yc,t,x which is 
a.s. strictly positive and so zc,t,x is admissible and zq^, x (s) = G(s,yG,t,x (s)) for s £ [t,T]. 
The same argument can be applied to the cases x < and x — 0. This means that, from 
Proposition 15. 51 for every fixed (t,x) £ [0, T] x R the couple {zG,t,x-,VG,t,x) (again with the 
notations of Proposition 15. 5J1 is optimal. Moreover the optimal control (state) is unique. In 
particular, when x > (x < 0) then we have a unique optimal couple where the optimal 
state is a.s. positive (negative). When x = the zero strategy is optimal. These facts cannot 
be immediately deduced using other verification theorems. Of course one could argue with 
ad hoc arguments applied to this problem but this is another story. We observe that this 
also implies that v is the value function. 

Remark 7.3 The procedure outlined in this example may be applied to other kind of prob- 
lems with bilinear state equation and current cost with growth 1 + ri (n G (0, 1)), eventually 
with infinite horizon. We think e.g. to optimal investment models where adjustment cost 
are not quadratic but of growth 1 + r\ (see e.g. \HEj )- Also one could treat similar models 
when no explicit solution can be found but it is possible to prove that there exists a strong 
C ' 1 solution of the HJB equation. 

Finally we mention two possible extensions of the problem introduced above that are 
interesting for economic applications (namely growth theory, optimal investment, optimal 
portfolio models): the case when state constraints arise and the case when the space gradient 
of the value function presents some singularities. Take for example the advertising example 
with rj < 0. These cases do not fall in our assumptions but we think that our procedure can 
be extended to cover them in a future work. I 
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8 Comparison with other verification results 

In this section we compare our results with other known verification techniques for non 
smooth solution of the HJB equation. A discussion about this matter has already been done 
in Section |21 Here we give a more detailed analysis. 

8.1 The classical result 

We start by recalling that the classical verification theorems for stochastic optimal control 
problems (see e.g. 13. p. 163]) adapted to our case are perfectly equal to our Theorems 14.91 
14. 161 except for the following facts: 

• they assume, in the finite horizon case, v € C 1 ' 2 ([0, T] x R") instead of v 6 C 0,1 ([0, T] x 

• they assume, in the infinite horizon case, v £ C 2 (R™) instead of v £ C 1 (R™). 

In Section El we have seen an example where v is surely C ' 1 ([0,T] x R n ) but it is not 
clear if it is also C 1 ' 2 ([0,T] x R"). Moreover, in Section[7|we have seen a one dimensional 
example where v is surely C - 1 ([0, T] x R) and not C 12 ([0, T] x M). 

8.2 The approximation result 

By approximation result we mean the verification theorem, proved e.g. in ^HJ in a special 
infinite dimensional case, that is proved using the approach that we have briefly recalled in 
Remark 14.41 The statement of such result is completely similar to our Theorems 14 . 91 or 14 . 1 61 
The only difference is that to prove this theorem one needs to know that the function v is 
a strong solution of HJB equation in a more restrictive sense (we may call them "stronger" 
solutions) pointed out in Remark 14.41 it is required that also the space derivative of the 
approximating sequence converges uniformly on compact sets, i.e. d x v n — » d x v. In the 
results of the present paper in place of this we have a weaker requirement, namely that, 
either Hypothesis 14.71 - (i) . or 14. 71 (ii) is satisfied. Regarding this point we note that: 

• in Application 1 the fact that the solution is strong follows directly from theorems on 
semigroups stated in jHOj- Here Hypothesis I4.7l (i) holds by the nondegeneracy of the 
diffusion coefficient while the convergence of the space derivative in such cases is not 
trivial at all and may be not true. 

• in Application 2, in the one dimensional example, as pointed out in Remark 17.21 it 
is straightforward to prove that u is a strong solution in the sense of Definition 14.61 
Here the diffusion coefficient is degenerate so to apply our results we need Hypothesis 
!4.7K 'ii'). This is not trivial to check in general but is in any case easier than the uniform 
convergence of the derivatives. In the special case where we calculate the value function 
the uniform convergence holds so also the approach of JSj mav be used. 

We finally point out that, to avoid the requirement of the convergence of the derivative 
on compact sets we need to use a completely different (and much more complex) approach 
based on the Fukushima-Dirichlet decomposition and on the representation result (Theorem 
14. 3|) proved in our companion paper |22j . 

8.3 The viscosity solution result 

When the HJB equation admits a viscosity solution it is possible to prove a verification 
theorem which is very general since it deals with only continuous solutions of HJB, and with 
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the case when also the diffusion coefficient is controlled, but which is less useful when we 
know that we have strong solutions. The theorem, in the model case studied in this paper, 
is the following (see e.g. 1231 1771 [39 for the proof and precise definition of superdifferentials 

,1,2,+ 
t+,x 



D t \'^v(s,y*(s)) used here) 



Theorem 8.1 Consider the problem £7J) - 0) with the following assumptions. The data 
Fo, F\, B, I, 4> are all uniformly continuous. Moreover they are Lipschitz continuous in the 
variable x uniformly with respect to the other variables. Moreover, for a suitable constant 
M > 0, 

\F (t, 0)| , \Fi (t, 0, z)\ , \B (t, 0)| , |i (i,0, z)\ , |0(O)| < M. 

Finally the control space U is a Polish space and the set of admissible controls Z a d (t) is 
given by all measurable and adapted processes on [t,T] with values in U. 

Let v £ C ([0, T] x MP) be a viscosity solution of the HJB equation Q). Then: 

• v(t,x) < V(t,x), for any (t,x) £ [0, T) x M™. 

• Let (y* (■) , z* (■)) be a given admissible pair for the problem starting at (t,x). Suppose 
that there exists (p*,q*, Q*) £ L% (t, T; R) x L 2 T (t, T; M") x L 2 T (t, T; S n ) such that for 
a.e.s G [t, T], 

( * / \ * / \ /^l* / \ \ -- T 

" 'I--. 

and 



( P *(s),q*(s),Q*( s )) e Dt££v(s,v*(s)), F -«- s - 



p*(s) - ^Tr [B* (s, x*(s)) Q* (s) B (s, x*(s))] - (F (s, x*(s)) , q* (,)) 

= H% v (s,x*(s),q*( S );z*(s)) = H ( S ,x*(s),q*( S )), F-a.s. 

then (y* (■) , z* (■)) is an optimal pair for the problem starting at (t,x). 

If we know that there exists a strong solution v 6 C 0,1 then to apply the above theorem 
we need to know that v is also a viscosity solution. This is not difficult as the concept of 
viscosity solution is more general in a wide range of cases (this is true since classical solutions 
are always viscosity solutions and limits of viscosity solutions are still viscosity solutions, 
see e.g. Q21); if so then we know that it must be 

q*(s) =d x v(s,y*(s)). 

The above sufficient condition states then that: 

if there exists (p*,q*,Q*) £ L%(t,T;R) x L%(t,T;R n ) x L%(t,T\S n ) such that for 
a.e.s £ [t,T], 

(p*( S ),q*( S ),Q*( S ))£D 1 t £+v( S ,y*(s)), P - a.s. 

(so q* (s) = d x v (s, y* (s))) and 

p*( S )-±Tr[B*( s ,x*(s))Q*(s)B(s,x*( S ))}-(F { s ,x*(s)),d x v(s,y*(s))) 

= H°, v ( S ,x*(s),d x v(s,y*(s));z*( S )) = H ( S ,x*(s),d x v( S ,y*(s))), P - a.s. 

then (2/*(-)i •£*(')) i s an optimal pair for the problem starting at (t,x). 
Our sufficient conditions instead states the following: 
if for a.e. s £ [t,T], 

H& v (s,x*(s),d x v(s,y* (s));z*(s)) = H° (s,x*(s),d x v(s,y* (s))) , F-a.s., 

then (y* (■) , z* (■)) is an optimal pair for the problem starting at (t,x). 
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The main difference is that in the Theorem 18. fl above we need to know about the exis- 
tence of a couple of processes p* and Q* (that play the role of time derivative and second 
space derivative) which are in fact not needed in our statement. This is not a trivial dif- 
ficulty in general. So our statement shows that, when strong solutions exist, one can get 
sharper sufficient conditions. Similar and even stronger considerations can be done for the 
necessary conditions and the existence of optimal feedbacks. In particular the analogous of 
the necessary condition stated by Proposition 15 . 21 in the viscosity setting says only that for 
every possible (p* (s) , q* (s) , Q* (s)) £ Dl££v(s,y*(s)), P - a.s. (so q* (s) = d x v (s, y* (s))) 
we have 

p*(t) - ^Tr [B* (t, x*(t)) Q* (t) B (t, x*(t))} - (F (t, x*(t)) , d x v (t, y* (*))) 

< H° cv (t,x*(t),d x v(t,y*(t));z*(t)) 

and this do not imply a priori that 

H° cv (t,x*(t),d x v (t,y* (t)) ;z*(t)) = H° (t,x*(t),d x v (t,y* (£))) , P - a.s 
since it may happen that 

p*(t) - ^Tr [B* (t, x*(t)) Q* (t) B (t, x*(t))} - (F (t, x*(t)) , d x v (t, y* (t))) 

< H°(t,x*(t),d x v(t,y*(t))) 

for all possible (p*(s),q*(s),Q*(s)) £ D t ^^v(s,y*(s)), P — a.s.. So, concerning necessary 
conditions we may say that the difference between our result and the viscosity solution result 
is wider. 

Remark 8.2 We must also note another point. The assumptions made in Theorem \8.1\ 
about the data are much more restricitve than our assumptions made for the model problem 
of Section We think that Theorem \8.1\ could be extended to more general situations but 
this extension is not trivial and not known at this stage. 

Of course the importance of the verification theorem for viscosity solutions is that it 
covers the cases when the control enters also in the diffusion coefficients and when the 
viscosity solutions has no further regularity. So it is in a sense natural that such result is 
more restrictive than ours when applied to cases when the control enters only in the drift 
and a strong solution exists. I 



8.4 The backward equations result 

This is a verification technique introduced for instance in |84| and |14j that is used when it 
is possible to find a mild solution to the HJB equation (in the sense recalled in Subsection 
I6.2|) and when it can be represented via the solution of a forward - backward system, see on 
this ^] |2] • The most recent and complete result is given in ^3] m the infinite dimensional 
case and we take the setting from it adapting the statement to our problem. Similar results 
in other context are given e.g. in [HI] (see also the references therein). 
Consider then the problem QJ - © with the following assumptions. 

1. The state space is X = R™ while the control space is U, a bounded subset of R m . 

2. The data Fo, B are continuously differentiable with bounded first derivative in the 
variable x uniformly with respect to t. 
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3. Ft (t, x,z)=B (i, a;) R (t, x) z where R : [0,T] x X ^ L (U, X) is continuously differ- 
entiable in x, bounded with its space derivative. 

4. Z is continuous and polynomially growing. 

5. 4> is continuously differentiable with bounded first derivatives. 

6. Defining the modified current value Hamiltonian K^ v (t, x, q;z) = I (t, x, z) + (q, -z) Rm 
we assume that K° = inf Kq v (i, x,q;z) is measurable, continuously differentiable 
in (x.p) with bounded derivative with respect to p and with polynomially growing 
derivative with respect to x. 

7. There exists a unique minimum point of the modified current value Hamiltonian Kq v 
given by T (i, x,p) where T is continuous. 

The result is the following. 

Theorem 8.3 Assume Hypotheses 1-7 above. Fix (t,x) 6 [0, T] x X. For all admissible 
control strategies z starting at (t, x) we have J(t, x; z) > v(t, x) and the equality holds if and 
only if the following feedback law is verified by z and y (•; t, x, z): 

z(s) = T(s, y (s) , R(s, y (s))*G(s, y (s))*d x v(s, V (*))), F-a.s. for a.e. s e [t, T\. (38) 

Finally there exists at least an admissible control strategy z starting at (i, x) for which ydtfj) 
holds. In such a system the closed loop equation: 

dy(s) = J F 1 o(s,y( S )) + J B( S ,y( S ))i?( S ,2;( S ))r 1 ( S , 2 ;( S ))d S 

+B(s,y(s))dW s , 8£[t,T\, (39) 

y (t) = x e X, 

where we set Ti (s, y (s)) = r (s, y (s) ,R(s,y (s))* B (s, y (s))* \7 x v (s, y (s))), admits a so- 
lution y and if z(s) = T (s, y (s) , R(s, y (s))*G(s, y (s))*V x v (s, y (s))) then the couple (z, y) 
is optimal for the control problem. 

We point out the following. 

• The dependence on the control in the state equation is linear and through the same 
operator B that drives the noise, moreover the space U is compact. This means that 
Hypothesis 14.71 (ii) is always satisfied. So the backward equation result Theorem 18.31 
cannot cover cases when only part (i) of Hypothesis 14.71 is satisfied, e.g. the one 
dimensional example of Sectional This is not a casual restriction but it seems to be a 
limitation due to the technique used in the proof (where Girsanov transform appears). 
It is not clear how to deal with more general cases as pointed out in ^] Section 7]. 
Moreover the linearity in the control says that, at the present stage, we cannot apply 
this technique to the cases described in Section [S] 

• The regularity assumptions on the coefficients are quite strong and in any case quite 
less general than our assumptions. In particular continuous differentiability is always 
required also in the Hamiltonian (assumption 6 above). 

• The result quoted above is proven for an infinite dimensional case where mild solutions 
exist and Girsanov theorem can be applied in a suitable way. Here we give a finite 
dimensional version of it. Of course the infinite dimensional case imposes certain 
restrictions and probably some technical assumptions may be removed. However some 
assumptions that strongly limits the applicability of the result (linearity in the control, 
smoothness of the coefficients) seem to constitute a structural limit of the technique 
used there. 
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